Visual communication systems and applications advocate increasingly larger screens and higher resolutions. This tendency is amplified with the appearance of larger CRT, LCD, PDP, projector High-Definition (HD) display systems such as in TVs, and larger digitally processed and stored visual information in the form of MPEG, DVD, DV, etc. As such, it becomes very important to improve the quality of images and videos that are displayed on large screens at high resolution. TV sets often implement video post-processors that improve and enhance the image/video signals to be displayed. Post-processors in such TV sets perform many functions including, but not limited to, image scaling, image noise reduction, image detail enhancement, and image color enhancement to improve and enhance the image/video signals for display.
Digital video content may be initially processed and encoded by a variety of digital compression techniques to overcome the problem of data bandwidth limitation in communication networks. The current Digital TV (DTV) broadcasting in the U.S. uses the MPEG-2 international video compression standard to compress digital video contents. DVD video contents are also processed by MPEG-2. High definition (HD) contents may be processed by MPEG-2 or MPEG-4. These MPEG processed digital videos contain a varying degree of artifacts that deteriorate the quality of displayed video images. The artifacts in MPEG-processed digital videos are referred to as “MPEG noise”, or “compression noise”, herein. As such, compression noise reduction, such as MPEG noise reduction, is one of the main functions implemented by a post-processor in a TV set. Compression noise reduction is a process that detects and removes such annoying MPEG noises from the digital videos before display on a display system screen.
There are different types of MPEG compression noises. One class of MPEG noises is known as block artifacts which are caused by the block-based image compressing schemes.
Block artifacts arise in images/videos that are compressed by block-based coding schemes such as JPEG, MPEG, and H.26X. In these coding schemes, a video image is divided into an array of N-by-N rectangular blocks (e.g., N=16) that are called macroblocks. Then, each macroblock is again sub-divided into M-by-M (e.g., M=8) sub-blocks. Each sub-block is typically processed by an M-by-M (e.g., 8-by-8) Discrete Cosine Transform (DCT), Quantization, Zig-zag scanning, and Entropy coding, independent of other sub-blocks. Because each sub-block (and each macroblock) is processed independently, a critical high-frequency portion of the image/video data that connects neighboring blocks is often lost and the superfluous edges and discontinuities appear at the block boundaries. Block artifacts are defined as appearances of these undesired superfluous edges or discontinuities at the block boundaries.
Block artifacts become more severe as the image/video is compressed more, i.e., at high compression rates. Also, scenes with high motion contents tend to contain more block artifacts.
The human visual system is extremely efficient at recognizing block artifacts. This is because humans have an extensive amount of visual knowledge and experience about what the world (objects and scenes) looks like. It is very easy, therefore, for humans to detect the artificially generated discontinuities and edges appearing across the picture at a regular interval. Even very small discrepancies are detected without much effort by a human viewer. On the other hand, machines lack the full-extent of visual knowledge that humans have. Specifically, conventional electronic devices or software programs that are built to detect and remove block artifacts rely only on very restricted inter-pixel, inter-block, or inter-frame relationships. Complete and accurate removal of block artifacts is therefore extremely difficult for such devices and programs. Many conventional devices/programs leave the block artifacts untreated or generate undesirable side-effects, such as blurring, as a result of inadequate processing mechanisms.
There is therefore a need for an image processing method and a system that collect and assemble, in an effective and intelligent manner, information about the contents of the image/video and effectively utilize such information to reduce/remove block artifacts in the compressed/coded image/video.